Monitoring mobile application performance

ABSTRACT

Aspects of the subject disclosure are directed towards monitoring application performance during actual use, particularly mobile application performance. Described is instrumenting mobile application binaries to automatically identify a critical path in user transactions, including across asynchronous-call boundaries. Trace data is logged by the instrumented application to capture UI manipulations, thread execution, asynchronous calls and callbacks, UI updates and/or thread synchronization. The trace data is analyzed to assist developers in improving application performance.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S.application Ser. No. 15/206,200, entitled “MONITORING MOBILE APPLICATIONPERFORMANCE,” filed Jul. 8, 2016 which is a continuation of and claimspriority to U.S. application Ser. No. 13/923,187, entitled “MONITORINGMOBILE APPLICATION PERFORMANCE,” filed Jun. 20, 2013, which areincorporated herein in their entirety.

BACKGROUND

There are currently over a million mobile applications in variousapplication marketplaces. Users rely on these applications for a widevariety of tasks, such as posting comments on social networking sites,conducting online banking and so forth.

The mobile-application marketplace is highly competitive. As a result,developers strive to maintain and improve the quality of theirapplications. Lab testing provides some guidance, but is seldomsufficient, as mobile applications are highly interactive and a fullrange of user interactions are difficult to simulate in a lab. Further,mobile applications experience a wide variety of environmentalconditions “in the field,” such as network connectivity (Wi-Fi or 3G),GPS-signal quality, and phone hardware, which can vary widely amongactual users. Other conditions also factor in, e.g., some platform APIschange their behavior depending on the battery level. Such diverseconditions are difficult to reproduce in a lab.

As a result, to improve the quality of an application, the applicationdeveloper/team needs to understand how an application performs in thefield. However, the asynchronous, multi-threaded nature of mobileapplications makes tracing application usage and application performanceto gather data difficult. The difficulties are compounded by theresource limitations inherent in the mobile platform, e.g., addingtracing code that assists in monitoring application performance cannotadversely impact the user experience.

SUMMARY

This Summary is provided to introduce a selection of representativeconcepts in a simplified form that are further described below in theDetailed Description. This Summary is not intended to identify keyfeatures or essential features of the claimed subject matter, nor is itintended to be used in any way that would limit the scope of the claimedsubject matter.

Briefly, various aspects of the subject matter described herein aredirected towards a technology in which application code is automaticallyinstrumented, including inserting one or more logging functions and oneor more event handlers into the application code to log events. Alsodescribed is inserting code into the application code to assign anidentifier for an invocation of an asynchronous call and detour thecallback provided to the asynchronous call to detour code to associatethe callback with the asynchronous call based upon the identifier.

In one aspect, an instrumenter is configured to instrument applicationcode with logging functionality and with code that associatesasynchronous calls with callbacks. An analysis component processed datalogged during execution of the application code via the loggingfunctionality into directed acyclic graphs representing one or moretransactions that occurred during execution.

In one aspect, during executing of instrumented application code, tracedata is logged via the instrumented application code. The trace datacorresponds to at least one of: user interface (UI) manipulations,thread execution, asynchronous calls and callbacks, UI updates or threadsynchronization. The trace data is uploaded for analysis processing.

Other advantages may become apparent from the following detaileddescription when taken in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example and not limitedin the accompanying figures in which like reference numerals indicatesimilar elements and in which:

FIG. 1 is a block diagram showing various example components includingan Instrumentation and analysis system that monitors mobile applicationperformance, according to one or more example embodiments.

FIG. 2 is a block diagram showing a structure of instrumenting of anapplication via logger and detour libraries, according to one or moreexample embodiments.

FIG. 3 is an example execution trace for example code showing a usertransaction, according to one or more example embodiments.

FIG. 4 is an example execution trace for example location-basedapplication code showing a user transaction and a critical path therein,according to one or more example embodiments.

FIG. 5 is an example execution trace showing an example applicationcrash/exception path, according to one or more example embodiments.

FIG. 6 is a flow diagram showing example steps related to matching anasynchronous call with a corresponding callback, according to one ormore example embodiments.

FIG. 7 is a representation of one example way in which an asynchronouscall may be detoured for matching the call with a correspondingcallback, according to one or more example embodiments.

FIGS. 8A and 8B are representations of directed acyclic graphs processedfrom logged trace data to represent user transactions, according to oneor more example embodiments.

FIG. 9 is a flow diagram showing example steps that may be taken toinstrument code, execute the instrumented code to collect trace data foranalysis, and analyze the trace data, according to one or more exampleembodiments.

FIG. 10 is a block diagram representing an example computing environmentin the form of a mobile computing device into which aspects of thesubject matter described herein may be incorporated.

DETAILED DESCRIPTION

Various aspects of the technology described herein are generallydirected towards monitoring mobile application performance, including byinstrumenting mobile application code (binaries) to collect trace data(traces). The traces are analyzed to automatically identify a criticalpath in user transactions, including across asynchronous-callboundaries, where in general user-perceived latency is set forth in theconcept of a critical path, in that a critical path provides thebottleneck path in a user transaction, such that changing the length ofany part of the critical path changes the user-perceived latency. Ingeneral, the critical path starts with a user manipulation event, andends with a UI update event. A user “transaction” as described hereinbegins with a user manipulation of the UI, and ends with completion ofthe synchronous and asynchronous tasks (threads) in the application thatwere triggered by the manipulation. As will be understood, manyintermediary actions may be taken during a transaction.

In one aspect, a lightweight instrumentation system is provided thatdoes not require input from the developer, nor any changes to theoperating system. In one implementation, the instrumentation mechanisminstruments mobile applications generally by interposing on eventhandlers. Performance data is collected in the field, and may beuploaded to a central server for offline analysis.

As will be understood, the instrumentation system described herein helpsapplication developers diagnose performance bottlenecks and failuresexperienced by their applications in the field. The instrumentationmechanism provides developers with information on a critical paththrough the code for user transactions. This information generally maybe used to point the developer to optimizations needed for improving theuser experience.

It should be understood that any of the examples herein arenon-limiting. As such, the present invention is not limited to anyparticular embodiments, aspects, concepts, structures, functionalitiesor examples described herein. Rather, any of the embodiments, aspects,concepts, structures, functionalities or examples described herein arenon-limiting, and the present invention may be used various ways thatprovide benefits and advantages in computing and imaging technology ingeneral.

FIG. 1 is a block diagram showing example concepts related to oneimplementation (e.g., an architecture) of an instrumentation andanalysis system 102 based upon the technology described herein. Theexemplified system 102 captures, with relatively minimal overhead, theinformation needed to build execution traces of user transactions andidentify their critical paths and exception paths. Note that in decidingwhat to capture, there is a tradeoff between the overhead and theability to give useful feedback to the developer.

As shown in FIG. 1, an application 104 (e.g., a binary) is instrumentedusing an instrumentation tool (an instrumenter 106) into an instrumentedapplication 108 as described herein. Note that while an applicationbinary is used to describe the technology, any code may benefit from thetechnology described herein, e.g., an operating system component, aservice, and so on may be similarly instrumented and analyzed, even ifnot technically considered an application.

In one implementation, the developer only needs to provide theinstrumenter 106 with application binaries; no other input or sourcecode annotation is needed. For a phone application, the instrumenter 106leverages the fact that phone applications are often written usinghigher-level frameworks and compiled to an intermediate language (bytecode). For example, one or more implementations are designed forapplications written using the Silverlight® framework (used by mostapplication in the Windows®-phone marketplace) and compiled to MSIL(Microsoft® Intermediate Language) byte code. MSIL preserves thestructure of the program, including types, methods and inheritanceinformation.

Users obtain instances of the instrumented application 108 from anapplication store 110. When users run the instrumented application 108on their devices 112 in the field, trace data 114 is collected anduploaded to a server 116. For example, in a Windows®-based phone, thebackground transfer service (BTS) may be used to upload the trace data114; in general, BTS uploads the data when no foreground applicationsare running, and provides a reliable transfer service in the face ofnetwork outages and losses.

A number of factors affect the performance of mobile application,including user input, environmental conditions, and so forth. Theapplication-execution trace may be captured in varying degrees ofdetail.

The trace data is analyzed by an analysis component 118, and thefindings/developer feedback 120 are made available to a developer/team122, e.g., via a web-based interface. The developer/team 122 may thenimprove the application, e.g., via an updated version. The knowledgelearned also may be able to improve other applications.

In general, the trace data 114 corresponds to a need to capture variouscategories of data, including when the user manipulates the UI; when theapplication code executes on various threads, and causality betweenasynchronous calls and callbacks. Other categories include data relatedto thread synchronization points (e.g., throughWait calls) and theircausal relationship, when the UI was updated, and any unhandledexceptions.

Additional data may be captured for analysis. For example, for certainasynchronous calls such as web requests and GPS calls, additionalinformation may be collected at the call and at the callback. As moreparticular examples, for web request calls, the URL and the networkstate may be logged; for GPS calls, the state of the GPS may be logged.The choice of the information to log may be guided by experience and thetradeoff between completeness and overhead. Actual data shows thatcritical paths in a user transaction often involve either network or GPSaccesses. By logging a small amount of additional information at certainpoints, additional meaningful feedback may be provided to the developer.

To collect the data, the application may be instrumented in varioussteps. In one implementation, the application binary is first read,assigning a unique identifier to methods in the application code and tosystem calls. Each call site is considered unique; e.g., if X is calledtwice, each call site gets its own identifier. This mapping is stored inmetadata (e.g., in a metadata file and uploaded to the analysiscomponent 118 for later use. Another step links libraries to theinstrumented application code 224, including a detour library 226 and alogger library 228, as generally represented in FIG. 2. The detourlibrary 226 (based on known technology) is dynamically generated duringinstrumentation, and exports a series of detouring functions that helpattribute callback executions to the asynchronous calls that triggeredthem. The logger library 228 exports logging functions and eventhandlers that insert trace records into a memory buffer 230. Each recordis tagged with a timestamp and the identifier of the thread that calledthe logging function. The memory buffer 230 is flushed to stable storageto prevent overflow as needed. When the application exits, the memorybuffer 230 is scheduled for upload using BTS.

Further, the application is instrumented with calls to methods in thelogger library 228 and detour library 226 from appropriate places tocollect the desired data. An example of this process is provided via thecode fragment shown below in Table 1, and the corresponding transactiondiagram in FIG. 3.

TABLE 1 void btnFetch_Click( object obj, RoutedEventArgs e) { var req =WebRequest.Create(url); req.BeginGetResponse(reqCallback, null); } voidreqCallback(IAsyncResult result) { /* Process */UIDispatcher.BeginInvoke(updateUI); } void updateUI( ) { /* Update UI */}

In general, the user inputs UI manipulation events, which are capturedwhen the user interacts with the UI (touch, flick, and so forth). Theunderlying framework 232 (FIG. 2) delivers the UI input events on the UIthread of the application running in the foreground. A first event inthis series is a ManipulationStarted event, and the last is aManipulationEnded event. Any application-specified handler to handle aUI event is also called on the UI thread in between these two events.For example, in the above code in Table 1, btnFetch Click handles theclick event for a button. When the user touches the button on thescreen, the handler is called in between the two manipulation events.

The logger library 228 exports handlers for ManipulationStarted andManipulationEnded events, which are added to the application's code, asgenerally shown below in the instrumented code of Table 2. The handlerslog the times of the events, which allows matching the UI manipulationto the correct application handler for that UI input.

TABLE 2 void btnFetch_Click( object obj, RoutedEventArgs e) { +Logger.LogUpcallStart(5); var req = WebRequest.Create(url); * Detour dt= DetourFactory.GetDetour(reqCallback, 7); * Logger.LogCallStart(7);req.BeginGetResponse(dt.Cb1, null); * Logger.LogCallEnd(7); +Logger.LogUpcallEnd(5); } void reqCallback(IAsyncResult result) { +Logger.LogUpcallStart(19); /* Process */ * Detour dt =DetourFactory.GetDetour(updateUI, 13); * Logger.LogCallStart(13);UIDispatcher.BeginInvoke(dt.Cb2); * Logger.LogCallEnd(13); +Logger.LogUpcallEnd(19); } void updateUI( ) { +Logger.LogUpcallStart(21); /* Update UI */ + Logger.LogUpcallEnd(21); }

Table 2 shows the instrumented version of the code in Table 1. Note thatthe actual instrumentation is done on MSIL byte code, however forreadability purposes, decompiled C# code is shown.

FIGS. 3-5 show typical asynchronous programming pattern used in mobileapplications. Mobile applications are UI-centric in nature and the UI ismanaged by a single, dedicated thread in which UI updates, and userinteractions with the UI take place on this thread. To maintain UIresponsiveness, applications avoid blocking the UI thread as much aspossible, and perform most work asynchronously. Some mobile-programmingframeworks (e.g., Silverlight®) do not provide synchronous APIs fortime-consuming operations like network I/O and location queries. Computetasks are typically carried out by spawning worker threads. Thus, userrequests are processed in a highly asynchronous manner. This isillustrated in FIG. 3, which graphically shows the execution trace forthe code snippet in Table 1. In FIG. 3, the horizontal line segmentsindicate time spent in thread execution, while arrows between linesegments indicate causal relationships between threads.

In FIG. 3, the circled numerals (0) through (8) corresponds to thefollowing description:

-   -   (0) the user starts the transaction by clicking a button;    -   (1) the OS invokes the event handler (btn-Fetch Click) in the        context of the UI thread;    -   (2) the handler makes an asynchronous HTTP request, providing        reqCallback as the callback;    -   (3) the handler quits, freeing the UI thread;    -   (4) time is spent downloading the HTTP content;    -   (5) when the HTTP request completes, the OS calls reqCallback in        a worker thread;    -   (6) the worker thread processes the fetched data;    -   (7) when the processing finishes, the worker thread invokes the        UI Dispatcher, to queue a UI update;    -   (8) the OS calls the dispatched function (updateUI)        asynchronously on the UI thread, which updates the UI.

In practice, applications are typically much more complex. For example,worker threads may in turn start their own worker threads, some userinteractions may start a timer to perform periodic tasks through thelifetime of an application, transactions may be triggered by sensorssuch as accelerometers, a user may interrupt a running transaction orstart another one in parallel, and so forth.

By way of example, FIG. 4 illustrates a pattern common to location-basedapplications. The application displays information about nearbyrestaurants and attractions to the user. A typical user transaction goesas follows. Upon user manipulation, the application asks the system toget a GPS fix, and supplies a callback to invoke when the fix isobtained. The system obtains the fix, and invokes the app-suppliedcallback in a worker thread at (2). The callback function reads the GPScoordinates and makes two parallel web requests to fetch somelocation-specific data. Then, the thread waits (4), for two completionsignals. The wait is indicated via a dotted line. As the two webrequests complete, the OS invokes their callbacks at (5) and (7). Thefirst callback signals completion to the blocked thread at (6), whilethe second one does so at (8). As a result of the second signal, theblocked thread wakes up at (9), and updates the UI via the dispatcher.

Given such complex behavior, it can be difficult for the developers toascertain where the bottlenecks in the code are and what optimizationsmight improve user-perceived responsiveness. In FIG. 4, the bottleneckpath involves the second web request, which took longer to complete.Note that these bottlenecks may be different for different users,depending on their device, location, network conditions, and usagepatterns.

Failure analysis is also complicated by the asynchronous nature of theapplication. Consider the example in FIG. 5. Suppose the applicationcrashes in the method parseURL( )(8), which is called in a worker threadthat started at parseXML( ) (7). Because the UI thread function thatstarted the web request has exited, the operating system has noinformation about the user context for this crash. Thus, in theexception log offered by contemporary popular mobile platforms, thedeveloper only sees a stack trace of the crashed thread, from parseURL() to parseXML( ) The developer however, may want more information, suchas the user manipulation that triggered the crash, to speed updebugging.

Described herein is a system that (among other advantages) can trackuser transactions across thread boundaries. The system helps developersunderstand the performance bottlenecks and failures experienced by theirapplications in the field. As will be understood, this is accomplishedby providing developers with critical paths for user transactions andexception paths when applications fail during a transaction.

Note that in FIG. 3, a user transaction starts when the usermanipulation occurs and ends when the updateUI method completes.However, a user transaction need not always end with a UI update. Forexample, a background task may continue processing past the UI update,without impacting user-perceived latency.

With respect to a critical path, In FIG. 3, the entire path from (0) to(8) constitutes the critical path of the transaction. The latency can bereduced either by reducing the download delay (4) or the processingdelay (6). In FIG. 4, the critical path is shown by the dotted boldarrowed lines. Note that in FIG. 4, activities related to the downloadand processing of the first web request are not on the critical path(because the web request 2 took the longest). The critical pathidentifies the portions of the code that directly impacts user-perceivedlatency.

However, the critical path may not always accurately characterize theuser experience. For example, a transaction may make multiple updates tothe UI (one after the other), and the user may care about only one ofthem, or the user may interrupt a transaction to start a new one. Thus,the critical path is useful for understanding performance bottlenecks.

To debug application failures, the system described herein provides thedeveloper exception paths. An exception path is the path from the usermanipulation to the exception method, spanning asynchronous boundaries.In FIG. 5, (0) to (8) is the exception path. The exception path pointsthe developer to the user manipulation that started the asynchronouspath leading to the crash.

With respect to capturing thread execution, the bold horizontal linesegments in FIG. 3 represent when the application code starts and endsexecuting on each thread. This can be determined from a full executiontrace that logs the start and end of every method. However, the overheadof capturing and uploading a full execution trace from a mobile phone isgenerally prohibitive. Thus, in one aspect, described herein issubstantially reducing the overhead by observing that at the beginningof each horizontal line segment in FIG. 3, the top frame in the thread'sstack corresponds to an application method (as opposed to a method thatis internal to the framework), and observing that this method is theonly application method on the stack. These methods are upcalls from theframework into the application code. It is sufficient to log the startand end of only upcalls. The upcalls are generated when the systeminvokes an application-specified handler (also called callback) methodfor any of various reasons, for example, to handle user input, timerexpiration, sensor triggers, or completion of I/O operations. Note thatspawning of worker threads involves upcalls; the application creates athread, and specifies a method as a start method. This method is invokedas a callback of Thread.Start at some later time.

To instrument the code, potential upcall methods may be identified usinga heuristic, namely when a method is specified as a callback to a systemcall, a reference to the method comprising a function pointer, (calleddelegate in .NET), is passed to the system call. For example, in Table1, a reference to reqCallback is passed to the BeginGetResponse systemcall. The MSIL code for creating a delegate has a fixed format, in whichtwo opcodes are used to push a function pointer onto the stack. Anymethod that is referenced by these opcodes may be called as an upcall;(note that certain UI handlers are passed to the system differently, andare identified as well).

The system captures the start and end times of potential upcalls, alongwith the identifiers assigned to them. As shown in Table 2, theinstrumentation added for tracking potential upcalls is prepended by‘+’. The methods in the example of Table 1 are potential upcalls andthus instrumented (Table 2). While this technique is guaranteed tocapture the upcalls, it may instrument more methods than necessary,imposing unnecessary overhead. In practice, this overhead is negligible,compared to the savings achieved.

Turning to matching asynchronous calls to the resulting upcalls, (i.e.,their callbacks), by way of example, in FIG. 3, the system needs tomatch labels two (2) and five (5). To do so, the system needs toidentify the call sites where an asynchronous system call was made;e.g., in Table 1, the BeginGetResponse call is an asynchronous systemcall. The system 102 logs when the callback started executing as anupcall, with the start of upcall execution tracked as described above.

To identify the call sites where an asynchronous system call was made,in general, the system 102 assumes that any system call that accepts adelegate as an argument is an asynchronous call. However, in practicethis general heuristic is refined, including by detecting system calls(e.g., using thread identifiers in the trace) that may invoke thesupplied callback synchronously. Another refinement is exemplified bythe “delayed callback” code in the following table:

TABLE 3 Thread t = new Thread(foo); ... ... t.Start( );

In this example, the callback delegate foo was specified when theconstructor was called, but is invoked only when Thread.Start is called,which may be much later. The general heuristic without refinementincorrectly matches the callback to the call site of the constructor,instead of Thread.Start. However, domain knowledge about the framework(e.g., Silverlight®) system libraries may be used, e.g., by knowing thatthe callback function is always invoked from Thread.Start. The systemlogs the identifier of the thread object at the constructor, and also atThread.Start. The object identifiers and the detour log described hereinallow matching the callback to the Thread.Start call. Eventsubscriptions may be similarly handled.

The system further needs to connect the beginning of callback executionto the corresponding asynchronous call. Note that a single callbackfunction (e.g., a completion handler for a web request) may be specifiedas a callback for several asynchronous system calls. Rewriting theapplication code to clone the callback function several times andassigning unique identifiers is not sufficient, because the asynchronouscall may be called in a loop (e.g., for each URL in a list, startdownload) specifying the same function as a callback. To handle suchscenarios, the callback methods are written to detour them through thedetour library 226; relevant detour code is shown in the followingtable, Table 4.

TABLE 4 public class DetourFactory { ... public static Detour GetDetour(Delegate d, int callId) { int matchId = getUniqueId( );Logger.LogAsyncStart(callId, matchId); return new Detour(d, matchId); }} public class Detour { int matchId; Delegate originalCb; publicDetour(Delegate d, int matchId) { this.originalCb = d; this.matchId =matchId; } public void Cb1(IAsyncResult result) {Logger.LogCallbackStart(this.matchId); Invoke(this.originalCb); } publicvoid Cb2( ) { ... } }

Note that in the instrumented code (Table 2), instrumentation used fordetour is tagged with ‘*’ (an asterisk).

Instrumentation may be added as follows, and as shown in FIG. 6:

-   -   (Steps 602 and 604) Identify the system call BeginGetResponse as        an asynchronous call. In this example, consider that the        instrumenter has assigned a call identifier of 7 to this call        site. The system logs the call site identifier, and the start        and end time of the call (step 604).    -   (Step 606) Generate a new method (e.g., referred to as cb1) that        matches the signature of the supplied callback function, i.e.,        reqCallback, and add it to the Detour class in the Detour        library. This method is responsible for invoking the original        callback (Table 6).    -   (Step 608) Instrument the call site to call GetDetour to        generate a new instance of the Detour object. This object stores        the original callback, and is assigned a unique ID (called        matchId) at runtime. This matchId helps match the asynchronous        call to the callback.    -   (Step 610) Rewrite the application code to replace the original        callback argument with the newly generated detour method,        Detour.cb1.

During execution of the instrumented code, the detour code is called asgenerally represented in FIG. 7. Via instrumentation, the asynchronouscall 772 is configured so that the callback is to the DetourObject 774,with the asynchronous call 772 associated with a MatchId. The callbackis to DetourCallback 776 of DetourObject 774, which matches the callbackto the call based upon the associated MatchId. From the detour object,the callback DownloadCallback(response) 778 proceeds as usual from theasynchronous call's perspective, but (via the detour) thelogging-related information is known to be associated with thisasynchronous call.

Note from Table 4 that the GetDetour method logs the beginning of anasynchronous call using the LogAsyncStart function of the Loggerlibrary. Similarly, the beginning of the callback is logged by theLogCallbackStart, which is called from cb1, just before the originalcallback is invoked. These records, and the UpcallStart record of theoriginal callback method are linked by the matchId, the call site ID,and their thread IDs, allowing the system to attribute the callback tothe correct asynchronous call.

Table 4 also shows another example of detouring. The UpdateUI method isa callback for the Beginlnvoke method of the UIDispatcher, and hence isdetoured.

A trace generated by the instrumented code in Table 2 is shown in aseries of records in Table 5.

TABLE 5 RecordId Records ThreadId 1 UIManipulationStarted 0 2MethodStart(5) 0 3 CallStart(7) 0 4 AsyncStart(7, 1) 0 5 CallEnd(7) 0 6MethodEnd(5) 0 7 UIManipulationEnded 0 8 CallbackStart(1) 1 9MethodStart(19) 1 10 CallStart(13) 1 11 AsyncStart(13, 2) 1 12CallEnd(13) 1 13 MethodEnd(19) 1 14 CallbackStart(2) 0 15MethodStart(21) 0 16 MethodEnd(21) 0 17 LayoutUpdated 0

Table 5 shows the trace generated by the instrumented code in Table 2.Records 1 and 7 show a UI Manipulation event. These records encompass anupcall (records 2-6) to the method btnFetch Click. As described above,this upcall is attributed to UI manipulation. This method makes theasynchronous system call BeginGetResponse (record 4), the callback ofwhich is detoured, and assigned a match ID of 1. Record 8 marks thebeginning of the execution of the detoured callback, and calls theactual callback method, reqCallback, which has a method ID of 19. Thismethod executes between records 9 and 13. Records 8 and 9 may be linkedbecause they have the same thread ID, and follow each other.

When reqCallback executes, another asynchronous call is made, namely acall to the UI dispatcher. The callback is detoured, and assigned amatch ID of 2. The actual callback method is UpdateUI, which has themethod ID of 21.

The completion of this method is indicated by record 16. Note that thismethod ran on the UI thread. Record 17 indicates that a LayoutUpdatedevent was triggered immediately after the execution of this method,which means that this method updated the UI.

In one implementation, the framework provides a set of methods forthread synchronization. The thread waits on a semaphore (e.g.,Monitor.Wait(obj)), and is woken up by signaling that semaphore (e.g.,Monitor.Pulse(obj)). Calls to these functions, and the identities ofsemaphore objects they use, are logged. The object identifiers can beused to determine the causal relationship between synchronization calls.Waiting on multiple objects, and thread join calls are handledsimilarly. Threads can also synchronize using shared variables asdescribed below.

With respect to capturing UI updates, in one implementation, theframework generates a LayoutUpdated event whenever the applicationfinishes updating the UI. In one implementation, if an upcall runs onthe UI thread (either event handlers, or application methods called viathe UIDispatcher), and updates one or more elements of the UI as part ofits execution, then a single LayoutUpdated event is raised when theupcall ends. The Logger library exports a handler for this event, whichis added to the application code. The handler logs the time this eventwas raised.

When an unhandled exception occurs in the application code, the systemterminates the application. Before terminating, the system delivers aspecial event to the application. The data associated with this eventcontains the exception type and the stack trace of the thread in whichthe exception occurred. To log this data, the logger library exports ahandler for this event, which is added to the application code.

Turning to analysis aspects, the traces are analyzed to delineateindividual user transactions, and to identify critical paths andexception paths. Transactions can also be analyzed in aggregate, tohighlight broader trends.

User transactions are represented by directed acyclic graphs. The graphis generated from the trace data. Consider the trace in Table 5, whichmay be converted to the graph in FIG. 8A. The graph contains five typesof nodes, namely: (M) User Manipulation, (S) Upcall start, (E) Upcallend, (A) Async call start, and (L) Layout updated. Each node representsone trace record (CallStart, CallEnd and CallBackStart records are usedfor bookkeeping, and are not mapped to nodes) and is identified by thetype and the record id. The mapping between node types M, S, E, A and Land the record types can be determined from Table 5. The edges betweennodes represent causal relationships. For example, theUIManipulationStarted event M1 triggers the start of the handler S2.Similarly, the start of callback execution S9 was caused by theasynchronous call A4. An upcall start node “causes” any subsequentactivity on that upcall. Hence shown are S2→A4, as the asynchronous callwas made during execution of the upcall, and S2→E6, to represent thefact that the upcall end is triggered by upcall start. Note that thegraph in FIG. 8A does not show any thread synchronization events; theseare represented by three types of nodes, namely (B) Thread blocked node,(F) Semaphore fired node, and (W) Thread wakeup node, which aredescribed below with reference to FIG. 8B.

When the application trace contains overlapping user transactions, thisapproach correctly separates them, and generates a graph for each. Datacorresponding to this graph representation may be used to discover thecritical path in a user transaction.

As described above, the critical path is the bottleneck path in the usertransaction. An algorithm for finding the critical path isstraightforward, as generally exemplified in FIG. 8A. The analysiscomponent 118 (FIG. 1) traverses the graph backwards, going from thelast UI update (L17), to the user manipulation event that signals thestart of the transaction (M1), traversing each directed edge in theopposite direction. This path, when reversed, yields the critical path:M1, S2, A4, S9, A11, S15, E16, L17. The example in FIG. 8A shows thatthe system correctly accounts for time spent inside upcalls, e.g., theedge (S9,E13) is not on the critical path, which means that any activityin the reqCallback method (Table 2), after calling the dispatcher, doesnot affect user-perceived latency. Note that refinements to thisalgorithm are discussed herein.

The transaction may update the UI multiple times. This results inmultiple L nodes in the transaction graph. Only the developer canaccurately determine which of these updates is significant. In suchcases, the instrumentation system, by default, reports the critical pathto the last L node. However, using a feedback interface or the like, thedeveloper can ask the instrumentation system to generate the criticalpath to any of the L nodes.

Thread synchronization via signaling is another aspect. The abovealgorithm implicitly assumes that each node has only one edge incidentupon it. This is not the case for the graph shown in FIG. 8B, whichrepresents the transaction shown in FIG. 8B. Node W, which is a threadwakeup node, has two edges incident upon it, because the thread waswaiting for two semaphores to fire (the two F nodes). In such cases, thesystem compares the timestamps of the semaphore-fire records, and picksthe later event. This yields the critical path shown in FIG. 8B by thebold dashed lines.

Note that an application may start a periodic timer, which fires atregular intervals and performs various tasks, including UI updates. Insome cases, periodic timers can also be used for thread synchronization.This pattern may be detected by the system, and then assume each timerfiring to be the start of a separate transaction. These transactions aretimer transactions, to distinguish them from user transactions. Thesetransactions may not end with UI updates. Sensor-driven transactions arehandled in a similar manner.

When the application crashes, the system logs the exception informationincluding the stack trace of the thread that crashed. Theinstrumentation mechanism-generated trace is known until that point. Thesystem walks the stack frames until a frame is found that contains themethod name of the last UpcallStart record in the instrumentationmechanism trace. The path from the start of the transaction to theUpcall start node, combined with the stack trace represents theexception path.

The instrumentation mechanism may help the developer see a “big picture”by analyzing the transactions in aggregate. There are various ways tolook at the aggregate data. For example, developers may benefit by usingthe aggregate data to uncover the root causes of performancevariability, and to discover “outliers”—e.g., transactions that tookabnormally long to complete compared to similar transactions. To performthis analysis, transactions with identical graphs may be groupedtogether, e.g., those having the same nodes and the same connectivity.These transactions represent the same user interaction with theapplication. This is a conservative grouping; the same user interactionmay occasionally generate different transaction graphs, but if twotransactions have the same graph, with a high probability theycorrespond to the same interaction.

While the transactions in a group have the same transaction graph, theircritical paths and durations can differ. To identify the major sourcesbehind this variability, a standard statistical technique calledAnalysis of Variance (ANOVA) may be used to help understand performancevariance. ANOVA quantifies the amount of variance in a measure that canbe attributed to individual factors that contribute to the measure.Factors may include network transfer, local processing and GPS querieswhich in turn can vary because of network type, device type, GPS state,user state, and so forth.

The system described herein also may flag outlier transactions to helpdevelopers identify performance bottlenecks. For example, transactionswith duration greater than (mean+(k*standard deviation)) in the groupmay be marked as outliers; k=3 in one implementation.

FIG. 9 is a summarization of example steps including instrumentation,execution and analysis steps performed by the instrumentation andanalysis system. Note that the instrumentation, execution and analysisare indirectly connected, e.g., a user may not download and/or executean instance of an instrumented application for a long time (e.g., days,weeks, months and so on) after instrumentation, and analysis may occur along time after execution. Steps 902 and 904 are directed toinstrumenting the application code, including adding the libraries (step902) and exporting the logging functions and event handlers (step 904).

Steps 910 and 912 are directed towards execution steps. As describedherein, during execution, the instrumented code logs data related to UImanipulations, thread execution, asynchronous calls and callbacks, UIupdates and/or thread synchronization. Asynchronous calls and callbacksare detoured to correctly associate the call with the callback asdescribed herein. Step 912 represents uploading the logged (trace) datato a server for subsequent analysis.

Steps 918 and 920 are directed to the analysis of the trace data,including receiving the trace data (which may be stored for lateranalysis). Analysis is represented by step 920, and includes conceptssuch as determining transactions, representing the transactions asdirected acyclic graphs (DAGs), and determining critical paths for thetransactions. Aggregation also may be performed to give the developer a“big picture” as described herein.

Example Operating Environment

FIG. 10 illustrates an example of a suitable mobile device 1000 on whichaspects of the subject matter described herein may be implemented. Themobile device 1000 is only one example of a device and is not intendedto suggest any limitation as to the scope of use or functionality ofaspects of the subject matter described herein. Neither should themobile device 1000 be interpreted as having any dependency orrequirement relating to any one or combination of components illustratedin the example mobile device 1000.

With reference to FIG. 10, an example device for implementing aspects ofthe subject matter described herein includes a mobile device 1000. Insome embodiments, the mobile device 1000 comprises a cell phone, ahandheld device that allows voice communications with others, some othervoice communications device, or the like. In these embodiments, themobile device 1000 may be equipped with a camera for taking pictures,although this may not be required in other embodiments. In otherembodiments, the mobile device 1000 may comprise a personal digitalassistant (PDA), hand-held gaming device, notebook computer, printer,appliance including a set-top, media center, or other appliance, othermobile devices, or the like. In yet other embodiments, the mobile device1000 may comprise devices that are generally considered non-mobile suchas personal computers, servers, or the like.

Components of the mobile device 1000 may include, but are not limitedto, a processing unit 1005, system memory 1010, and a bus 1015 thatcouples various system components including the system memory 1010 tothe processing unit 1005. The bus 1015 may include any of several typesof bus structures including a memory bus, memory controller, aperipheral bus, and a local bus using any of a variety of busarchitectures, and the like. The bus 1015 allows data to be transmittedbetween various components of the mobile device 1000.

The mobile device 1000 may include a variety of computer-readable media.Computer-readable media can be any available media that can be accessedby the mobile device 1000 and includes both volatile and nonvolatilemedia, and removable and non-removable media. By way of example, and notlimitation, computer-readable media may comprise computer storage mediaand communication media. Computer storage media includes volatile andnonvolatile, removable and non-removable media implemented in any methodor technology for storage of information such as computer-readableinstructions, data structures, program modules, or other data. Computerstorage media includes, but is not limited to, RAM, ROM, EEPROM, flashmemory or other memory technology, CD-ROM, digital versatile disks (DVD)or other optical disk storage, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage devices, or any othermedium which can be used to store the desired information and which canbe accessed by the mobile device 1000.

Communication media typically embodies computer-readable instructions,data structures, program modules, or other data in a modulated datasignal such as a carrier wave or other transport mechanism and includesany information delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media includes wired media such as awired network or direct-wired connection, and wireless media such asacoustic, RF, Bluetooth®, Wireless USB, infrared, Wi-Fi, WiMAX, andother wireless media. Combinations of any of the above should also beincluded within the scope of computer-readable media.

The system memory 1010 includes computer storage media in the form ofvolatile and/or nonvolatile memory and may include read only memory(ROM) and random access memory (RAM). On a mobile device such as a cellphone, operating system code 1020 is sometimes included in ROM although,in other embodiments, this is not required. Similarly, applicationprograms 1025 are often placed in RAM although again, in otherembodiments, application programs may be placed in ROM or in othercomputer-readable memory. The heap 1030 provides memory for stateassociated with the operating system 1020 and the application programs1025. For example, the operating system 1020 and application programs1025 may store variables and data structures in the heap 1030 duringtheir operations.

The mobile device 1000 may also include other removable/non-removable,volatile/nonvolatile memory. By way of example, FIG. 10 illustrates aflash card 1035, a hard disk drive 1036, and a memory stick 1037. Thehard disk drive 1036 may be miniaturized to fit in a memory slot, forexample. The mobile device 1000 may interface with these types ofnon-volatile removable memory via a removable memory interface 1031, ormay be connected via a universal serial bus (USB), IEEE 10394, one ormore of the wired port(s) 1040, or antenna(s) 1065. In theseembodiments, the removable memory devices 1035-437 may interface withthe mobile device via the communications module(s) 1032. In someembodiments, not all of these types of memory may be included on asingle mobile device. In other embodiments, one or more of these andother types of removable memory may be included on a single mobiledevice.

In some embodiments, the hard disk drive 1036 may be connected in such away as to be more permanently attached to the mobile device 1000. Forexample, the hard disk drive 1036 may be connected to an interface suchas parallel advanced technology attachment (PATA), serial advancedtechnology attachment (SATA) or otherwise, which may be connected to thebus 1015. In such embodiments, removing the hard drive may involveremoving a cover of the mobile device 1000 and removing screws or otherfasteners that connect the hard drive 1036 to support structures withinthe mobile device 1000.

The removable memory devices 1035-1037 and their associated computerstorage media, discussed above and illustrated in FIG. 10, providestorage of computer-readable instructions, program modules, datastructures, and other data for the mobile device 1000. For example, theremovable memory device or devices 1035-1037 may store images taken bythe mobile device 1000, voice recordings, contact information, programs,data for the programs and so forth.

A user may enter commands and information into the mobile device 1000through input devices such as a key pad 1041 and the microphone 1042. Insome embodiments, the display 1043 may be touch-sensitive screen and mayallow a user to enter commands and information thereon. The key pad 1041and display 1043 may be connected to the processing unit 1005 through auser input interface 1050 that is coupled to the bus 1015, but may alsobe connected by other interface and bus structures, such as thecommunications module(s) 1032 and wired port(s) 1040. Motion detection1052 can be used to determine gestures made with the device 1000.

A user may communicate with other users via speaking into the microphone1042 and via text messages that are entered on the key pad 1041 or atouch sensitive display 1043, for example. The audio unit 1055 mayprovide electrical signals to drive the speaker 1044 as well as receiveand digitize audio signals received from the microphone 1042.

The mobile device 1000 may include a video unit 1060 that providessignals to drive a camera 1061. The video unit 1060 may also receiveimages obtained by the camera 1061 and provide these images to theprocessing unit 1005 and/or memory included on the mobile device 1000.The images obtained by the camera 1061 may comprise video, one or moreimages that do not form a video, or some combination thereof.

The communication module(s) 1032 may provide signals to and receivesignals from one or more antenna(s) 1065. One of the antenna(s) 1065 maytransmit and receive messages for a cell phone network. Another antennamay transmit and receive Bluetooth® messages. Yet another antenna (or ashared antenna) may transmit and receive network messages via a wirelessEthernet network standard.

Still further, an antenna provides location-based information, e.g., GPSsignals to a GPS interface and mechanism 1072. In turn, the GPSmechanism 1072 makes available the corresponding GPS data (e.g., timeand coordinates) for processing.

In some embodiments, a single antenna may be used to transmit and/orreceive messages for more than one type of network. For example, asingle antenna may transmit and receive voice and packet messages.

When operated in a networked environment, the mobile device 1000 mayconnect to one or more remote devices. The remote devices may include apersonal computer, a server, a router, a network PC, a cell phone, amedia playback device, a peer device or other common network node, andtypically includes many or all of the elements described above relativeto the mobile device 1000.

Aspects of the subject matter described herein are operational withnumerous other general purpose or special purpose computing systemenvironments or configurations. Examples of well known computingsystems, environments, and/or configurations that may be suitable foruse with aspects of the subject matter described herein include, but arenot limited to, personal computers, server computers, hand-held orlaptop devices, multiprocessor systems, microcontroller-based systems,set top boxes, programmable consumer electronics, network PCs,minicomputers, mainframe computers, distributed computing environmentsthat include any of the above systems or devices, and the like.

Aspects of the subject matter described herein may be described in thegeneral context of computer-executable instructions, such as programmodules, being executed by a mobile device. Generally, program modulesinclude routines, programs, objects, components, data structures, and soforth, which perform particular tasks or implement particular abstractdata types. Aspects of the subject matter described herein may also bepracticed in distributed computing environments where tasks areperformed by remote processing devices that are linked through acommunications network. In a distributed computing environment, programmodules may be located in both local and remote computer storage mediaincluding memory storage devices.

Furthermore, although the term server may be used herein, it will berecognized that this term may also encompass a client, a set of one ormore processes distributed on one or more computers, one or morestand-alone storage devices, a set of one or more other devices, acombination of one or more of the above, and the like.

CONCLUSION

While the invention is susceptible to various modifications andalternative constructions, certain illustrated embodiments thereof areshown in the drawings and have been described above in detail. It shouldbe understood, however, that there is no intention to limit theinvention to the specific forms disclosed, but on the contrary, theintention is to cover all modifications, alternative constructions, andequivalents falling within the spirit and scope of the invention.

What is claimed is:
 1. A method, comprising, automatically instrumentingapplication code, including inserting one or more logging functions andone or more event handlers into the application code to log events, andinserting code into the application code to assign an identifier for aninvocation of an asynchronous call and detour the callback provided tothe asynchronous call to detour code to associate the callback with theasynchronous call based upon the identifier.
 2. The method of claim 1wherein inserting the one or more logging functions and the one or moreevent handlers into the application code comprises adding a loggerlibrary to the application code that exports one or more loggingfunctions and one or more event handlers.
 3. The method of claim 1further comprising adding a detour library to provide the detour code.4. The method of claim 1 further comprising, logging events for a usertransaction.
 5. The method of claim 4 further comprising, analyzing theuser transaction, including identifying a critical path therein.
 6. Themethod of claim 5 further comprising, representing the user transactionas a directed acyclic graph, and wherein identifying the critical pathcomprises processing the directed acyclic graph.
 7. The method of claim1 further comprising, logging events including timing informationrelated to at least one of the events.
 8. The method of claim 1 furthercomprising, tracking an exception path.
 9. The method of claim 1 furthercomprising, logging events during application execution, includinglogging at least one of: user interface (UI) manipulations, threadexecution, asynchronous calls and callbacks, UI updates or threadsynchronization.
 10. The method of claim 9 wherein logging threadexecution comprises identifying upcalls.
 11. A system comprising, aninstrumenter, the instrumenter configured to instrument application codewith logging functionality and with code that associates asynchronouscalls with callbacks, and analysis component, the analysis componentconfigured to process data logged during execution of the applicationcode via the logging functionality into directed acyclic graphsrepresenting one or more transactions that occurred during execution.12. The system of claim 11 wherein the analysis component is configuredto process at least one of the directed acyclic graphs to determine acritical path.
 13. The system of claim 11 wherein the analysis componentis configured to process logged data into an exception path.
 14. Thesystem of claim 11 wherein the logging functionality is configured tolog data related to at least one of: a user transaction, a timertransaction or a sensor-driven transaction.
 15. The system of claim 11wherein the logging functionality is configured to log data related toat least one of: user interface (UI) manipulation, thread execution,asynchronous calls and callbacks, UI updates or thread synchronization.16. The system of claim 11 wherein the application code is executable ona smartphone or a tablet computer, or both.
 17. One or moremachine-readable storage media having executable instructions, whichwhen executed perform steps, comprising: (a) executing instrumentedapplication code; (b) logging data during execution via the instrumentedapplication code, the data corresponding to at least one of: userinterface (UI) manipulations, thread execution, asynchronous calls andcallbacks, UI updates or thread synchronization; and (c) uploading thedata for analysis processing.
 18. The one or more machine-readablestorage media of claim 17 wherein logging the data during executioncomprises matching an asynchronous call to a corresponding callback. 19.The one or more machine-readable storage media of claim 17 whereinmatching the asynchronous call to a corresponding callback comprisesInstrumenting the asynchronous call to assign an unique identifier anddetour the callback provided to the asynchronous call to call back todetour code, and associating the callback with the asynchronous call viathe identifier.
 20. The one or more machine-readable storage media ofclaim 17 having further executable instructions comprising, analyzingthe data, including processing transactions logged within the data intodirected acyclic graphs, and determining a critical path for atransaction by processing the directed acyclic graphs.